CRITICAL ANALYSIS OF ELECTRIC VEHICLES IN CANADA¶
Introduction¶
What is our dataset about?¶
We have selected 5 data sets, hoping to understand the fuel efficiency and the suitability in different situations among certain vehicle types.¶
What are we trying to prove here?¶
Where is our dataset coming from?¶
Vehicle Sales Data :¶
Data Collection and Look Up¶
In [1]:
import pandas as pd
import pandas_profiling
import numpy as np
import matplotlib.pyplot as plt
%pylab inline
pylab.rcParams['figure.figsize'] = (12.0, 10.0)
import seaborn as sns
%matplotlib inline
from pandas.plotting import scatter_matrix
from scipy import stats
from scipy import stats, special
from sklearn import model_selection, metrics, linear_model, datasets, feature_selection
from seaborn import pairplot, heatmap
from sklearn.model_selection import train_test_split
# Importing All needed Libraries
Reading files into pandas¶
In [2]:
vs = pd.read_excel('VehiclesSalesData.xls')
# Vehicles Sales is vs now to be easily recalled.
In [3]:
ev = pd.read_csv('2012-2020_Battery_Electric_Vehicles.csv')
# Battery Electric Vehicles is ev now to be easily recalled.
In [4]:
phev = pd.read_csv('2012-2020_Plug-in_Hybrid_Electric_Vehicles.csv')
# Plug in Hybrid Electric Vehicles is phev now to be easily recalled.
Take a look at the data and the first few rows¶
In [5]:
print("Vehicles Sales rows and columns:")
print(vs.shape)
print("Battery Electric Vehicles rows and columns:")
print(ev.shape)
print("Plug in Hybrid Electric Vehicles rows and columns:")
print(phev.shape)
# Number of rows and columns.
In [6]:
print("Vehicles Sales First & Last 5 rows:")
print(vs.head)
print("Battery Electric Vehicles First & Last 5 rows:")
print(ev.head)
print("Plug in Hybrid Electric Vehicles First & Last 5 rows:")
print(phev.head)
# First and last 5 rows for each data.
Descriptive Statistics¶
In [7]:
vs.describe(include='all')
# View statistical details.
Out[7]:
In [8]:
ev.describe(include='all')
# View statistical details.
Out[8]:
In [9]:
phev.describe(include='all')
# View statistical details.
Out[9]:
Data cleanup (handling missing values, duplicates, errors, outliers)¶
In [10]:
vs.isnull().values.any() # Any null values?
Out[10]:
In [11]:
ev.isnull().values.any() # Any null values?
Out[11]:
In [12]:
phev.isnull().values.any() # Any null values?
Out[12]:
In [13]:
vs.isnull().sum() # Where are the null values?
Out[13]:
In [14]:
ev.isnull().sum() # Where are the null values?
Out[14]:
In [15]:
phev.isnull().sum() # Where are the null values?
Out[15]:
In [16]:
vs.duplicated().any() # Any duplicated values?
Out[16]:
In [17]:
ev.duplicated().any() # Any duplicated values?
Out[17]:
In [18]:
phev.duplicated().any() # Any duplicated values?
Out[18]:
In [19]:
vs.info()
# Information about the data including the index, data type in columns.
In [20]:
ev.info()
# Information about the data including the index, data type in columns.
In [21]:
phev.info()
# Information about the data including the index, data type in columns.
In [22]:
vs.describe() # Looking for outliers.
Out[22]:
In [23]:
ev.describe() # Looking for outliers.
Out[23]:
In [24]:
phev.describe() # Looking for outliers.
Out[24]:
Let's Clean¶
Vehicle Sales Cleanup¶
In [25]:
vs
Out[25]:
In [26]:
vs.fillna(0, inplace=True)
# Replace all blank spaces with zero or dropping the rows(to make visualization simple).
# Because replacing blank spaces with the mean value will make the sales count inaccurate.
In [27]:
vs.drop([
'DGUID', 'Statistics', 'UOM', 'UOM_ID', 'SCALAR_FACTOR', 'SCALAR_ID',
'VECTOR', 'COORDINATE', 'STATUS', 'SYMBOL', 'TERMINATED', 'DECIMALS'
],
axis=1,
inplace=True)
# Drop most of the columns as there is no difference or effect on our data.
# Keeping only the number of units sold per vehicle and fuel type in certain locations from 2011 to 2019.
In [28]:
vs.columns # View all Columns names before renaming.
Out[28]:
In [29]:
vs.columns = [
'Date', 'Location', 'Fuel_Type', 'Vehicle_type', 'Number_of_Cars_sold'
]
# Changing the columns names to be more clear.
In [30]:
vs = vs[(vs['Fuel_Type'] != 'Diesel') & (vs['Fuel_Type'] != 'All fuel types')]
vs = vs[(vs['Fuel_Type'] != 'Other fuel types')
& (vs['Fuel_Type'] != 'Gasoline')]
# Drop some rows with inefficient fuel types(gasoline and diesel) and other fuel types(liquid propane).
In [31]:
vs # Let's have a look....is it readable now?
Out[31]:
Electric Vehicles Cleanup¶
In [32]:
ev = pd.read_csv('2012-2020_Battery_Electric_Vehicles.csv', header=[0, 1])
ev.columns = ev.columns.map('_'.join)
# Combining the headers with the first row.
In [33]:
ev.replace(NaN, 0) # Replace all NAN with 0.
Out[33]:
In [34]:
ev.columns # View all Columns names before renaming.
Out[34]:
In [35]:
ev.columns = [
'Year', 'Make', 'Model', 'Vehicle Class', 'Motor(kW)', 'Transmission',
'Fuel Type', 'City(kWh/100 km)', 'Highway(kWh/100 km)',
'City+Highway(kWh/100 km)', 'City(Litre/100 km)', 'Highway(Litre/100 km)',
'City+Highway(Litre/100 km)', 'Range(km)', 'CO2 Emissions(g/km)',
'CO2 Rating', 'Smog Rating', 'Recharge_Time(hours)'
]
# Changing the columns names to be more clear.
In [36]:
ev.drop([
'Transmission', 'Fuel Type', 'City(Litre/100 km)', 'Highway(Litre/100 km)',
'City+Highway(Litre/100 km)', 'CO2 Emissions(g/km)', 'CO2 Rating',
'Smog Rating'
],
axis=1,
inplace=True)
# Drop Transmission, Fuel Type(All cars are electirc), Anything in Litre, all emissions as there is no difference or effect on our data.
In [37]:
ev # Let's have a look....is it readable now?
Out[37]:
Plug-in Hybrid Electric Vehicles Cleanup¶
In [38]:
phev
Out[38]:
In [39]:
phev = pd.read_csv('2012-2020_Plug-in_Hybrid_Electric_Vehicles.csv',
header=[0, 1])
phev.columns = phev.columns.map('_'.join)
# Combining the headers with the first row.
In [40]:
phev.columns = [
'Year', 'Make', 'Model', 'Vehicle Class', 'Motor(kW)',
'Engine Size(Litre)', 'Cylinders', 'Transmission',
'Fuel Type(Electricity/Gasoline)', 'Electricity/Gasoline(100km)',
'Range(km)', 'Recharge Time(hours)', 'Fuel Type 2(Gasoline only)',
'City(Litre/100km)', 'Highway(Litre/100km)', 'Highway&City(Litre/100km)',
'Combined_Range(km)', 'CO2 Emissions(g/km)', 'CO2 Rating', 'Smog Rating'
]
# Changing the columns names to be more clear.
In [41]:
phev.drop(['Transmission'], axis=1, inplace=True)
# Drop Transmission as there is no difference or effect on our data.
In [42]:
phev['CO2 Rating'].fillna(0, inplace=True)
# Replace all NAN with 0.
In [43]:
phev['Smog Rating'].fillna(0, inplace=True)
# Replace all NAN with 0.
In [44]:
phev # Let's have a look....is it readable now?
Out[44]:
In [45]:
vs.to_csv('vs.csv')
ev.to_csv('ev.csv')
phev.to_csv('phev.csv')
# Saving the cleaned data for visulaization.
Data Visualization¶
Vehicle Sales Visualization¶
In [46]:
vsp = pandas_profiling.ProfileReport(vs)
# Pandas Profiling for Vehicle Sales
In [47]:
vsp.to_notebook_iframe()
# Pandas profiling interactive animation on the notebook(Html viewing)
In [48]:
sns.heatmap(vs.corr())
plt.title('Vehicle Sales');
# Visulazing the correlation on a heatmap.
In [49]:
sns.barplot(x="Number_of_Cars_sold", y="Location", hue="Fuel_Type", data=vs)
plt.xlabel('Units Sold')
plt.ylabel('Province')
plt.title('Number of cars sold in each location across Canada');
# Plotting using the barplot (How many car sold in each province?)
In [50]:
sns.pointplot(x="Number_of_Cars_sold", y="Location", hue="Fuel_Type", data=vs)
plt.xlabel('Units Sold')
plt.ylabel('Province')
plt.title('Number of cars sold in each location across Canada');
#Plotting on a pointplot (How many car sold in each province?)
In [51]:
plt.scatter(x=vs['Number_of_Cars_sold'],\
y=vs['Fuel_Type'],\
alpha=0.5,\
s=50,\
color='g',\
marker='x')
plt.xlabel('Units Sold')
plt.ylabel('Fuel Type')
plt.title('Number of cars sold per fuel type');
# Matplotlib (How many cars sold for each fuel type?)
In [52]:
# The whole data on a scatter matrix.
scatter_matrix(vs, alpha=0.5, figsize=(12, 12), diagonal='kde', s=100)
plt.title('Number of cars sold per year');
# Scatter Matrix (How many cars sold per calendar year?)
In [53]:
sns.factorplot(x="Number_of_Cars_sold",
y="Vehicle_type",
hue="Fuel_Type",
data=vs,
col="Date",
kind="box",
height=3.5,
aspect=1.5)
plt.title('Cars sold by vehicle and fuel type over the years');
# Plotting using factorplot with specific size and aspect.(Number of cars sold by vehicle type per calendar year)
In [54]:
g = sns.PairGrid(vs,
x_vars=["Number_of_Cars_sold", "Date"],
y_vars=["Fuel_Type", "Location"],
aspect=1.75,
height=2.25)
g.map(sns.violinplot, palette="Blues_r")
plt.title('Cars sold over the years for different fuel type among Canada');
# Showing categorical relationship on a pair grid & a violinplot.
Electric Vehicles Visualization¶
In [55]:
pev = pandas_profiling.ProfileReport(ev)
# Pandas Profiling for Electric Vehicle.
In [56]:
pev.to_notebook_iframe()
# Pandas profiling interactive animation on the notebook(Html viewing)
In [57]:
sns.heatmap(ev.corr())
plt.title('Electric vehicle')
# Visulazing the correlation on a heatmap.
Out[57]:
In [58]:
plt.scatter(x=ev['Range(km)'],\
y=ev['Recharge_Time(hours)'],\
alpha=0.5,\
s=50,\
color='b',\
marker='x')
plt.xlabel('Range per Km')
plt.ylabel('Charging Time hourly')
plt.title('Range by Km for hourly charging')
# Matplotlib (How long can the car go with one charge?)
Out[58]:
In [59]:
sns.factorplot(x="Recharge_Time(hours)",
y="Range(km)",
data=ev,
col="Make",
kind="box",
height=3.5,
aspect=1.5)
plt.title('Range by Km for hourly charging for each car make');
# Plotting using factorplot with specific size and aspect.(How long can each model go with one charge?)
Plug-in Hybrid Electric Vehicles Visualization¶
In [60]:
sns.heatmap(phev.corr())
plt.title('Plug-in Hybrid Electric Vehicle')
# Visulazing the correlation on a heatmap.
Out[60]:
In [61]:
sns.barplot(x="Combined_Range(km)", y="Make", data=phev)
plt.xlabel('Range by km for hybrid vehicles')
plt.ylabel('Make of the car')
plt.title('Range by make of Hybrid cars');
# Plotting using the barplot (Which car can go longer?)
Train/Test Split¶
In [62]:
train1, test1 = model_selection.train_test_split(vs)
In [63]:
train2, test2 = model_selection.train_test_split(ev)
In [64]:
train3, test3 = model_selection.train_test_split(phev)
In [65]:
len(test1) / (len(train1) + len(test1))
Out[65]:
In [66]:
len(test2) / (len(train2) + len(test2))
Out[66]:
In [67]:
len(test3) / (len(train3) + len(test3))
Out[67]:
In [68]:
vs = vs.sample(frac=1)
In [69]:
ev = ev.sample(frac=1)
In [70]:
phev = phev.sample(frac=1)
In [71]:
train1
Out[71]:
In [72]:
test1
Out[72]:
In [73]:
train2
Out[73]:
In [74]:
test2
Out[74]:
In [75]:
train3
Out[75]:
In [76]:
test3
Out[76]:
Train the model using training data¶
In [77]:
slope, intercept, r_value, p_value, slope_std_error = stats.linregress(
train1.index, train1['Number_of_Cars_sold'])
In [78]:
train_regr = intercept + slope * train1.index
scatter(train1.index, train1['Number_of_Cars_sold'])
plt.plot(train1.index, train_regr, color='red')
Out[78]:
In [79]:
slope, intercept, r_value, p_value, slope_std_error = stats.linregress(
train2.index, train2['Range(km)'])
In [80]:
train_regr = intercept + slope * train2.index
scatter(train2.index, train2['Range(km)'])
plt.plot(train2.index, train_regr, color='red')
Out[80]:
In [81]:
slope, intercept, r_value, p_value, slope_std_error = stats.linregress(
train3.index, train3['Combined_Range(km)'])
In [82]:
train_regr = intercept + slope * train3.index
scatter(train3.index, train3['Combined_Range(km)'])
plt.plot(train3.index, train_regr, color='red')
Out[82]:
In [83]:
!jupyter nbconvert EV.ipynb --to 'html'